Special Issue: Critical assessment issues in work-integrated learning 


ePortfolios as evidence of standards and outcomes in 
work-integrated learning 

SONIA FERNS 1 

JUDE COMFORT 

Curtin University, Perth, Australia 


Electronic portfolios (ePortfolios) are a student focused tool which support and evidence work-integrated learning 
(WIL) experiences and capabilities in a tertiary education setting. Such settings are increasingly faced by a regulatory 
framework requiring evidence of student competency and skill acquisition. The commitment of educational institutions 
to integrate WIL in programs of study has driven the development of electronic portfolios using various platforms. 
This paper will discuss how the ePortfolio has evolved in response to the WIL agenda and its potential to capture 
authentic assessment and evidence competencies far beyond the traditional academic transcript. The value of an 
electronic repository of artefacts for students in the transition from institution to employment is highlighted, and the 
potential benefits of employers' involvement in the future direction of ePortfolios are discussed. A case study approach 
is employed to articulate the possibilities and challenges of achieving institutional acceptance, support and 
implementation of ePortfolios. (Asia-Pacific Journal of Cooperative Education, Special Issue, 2014,15(3), 269-280) 
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Electronic portfolios (ePortfolios) are gaining increased attention as a mechanism for 
students to collate and showcase artifacts which provide evidence of skills and attributes 
acquired through curricula and extra-curricular experiences (Herring & Notar, 2011). 
Furthermore, as universities transition into a standards-based, regulatory framework, the 
need to demonstrate student outcomes will become increasingly important as an 
accountability measure for employers and other stakeholders (Department Education, 
Employment and Workplace Relations, 2011). While work-integrated learning (WIL) is 
acknowledged as a strategy for integrating theory within a practice-based context, assessing 
WIL presents multiple challenges given its multi-dimensional and complex nature (Knight & 
Yorke, 2004). Developing workplace proficiency is dependent on a range of variables which 
include the nature of the work environment; personal self-efficacy; and individual skill sets. 
Arguably, evidencing the acquisition of workplace skills compounds the complexity given 
the traditional assessment protocols in universities which do not auger well for judging 
variable student outcomes (Hodges, 2011). 

VALUE OF EPORTFOLIOS 

Technology is having a profound effect on all facets of society and functions of everyday life. 
Arguably, the digital era is having the greatest impact on education where it is challenging 
traditional teaching and learning practices and approaches (Williams, 2011). Higher 
education institutions have an obligation to prepare students with lifelong learning skills and 


Corresponding author: Sonia Ferns, s.ferns@curtin.edu.au 






FERNS, COMFORT: ePortfolios as evidence of standards and outcomes in WIL 


the global economy. Proficiency with technology clearly sits within the realm of skills 
necessary to be a functional and contributing member of society. The ePortfolio is 
instrumental to achieving this vision. 

There are multiple benefits of using an ePortfolio to showcase artefacts which validate 
student learning outcomes in a WIL context (Voigt, 2009). Firstly, they provide incremental 
evidence through-out a degree program of different developmental phases and individual 
growth thereby demonstrating student work-readiness. The ePortfolio is an interactive 
platform that enables students to recognize personal strengths in skill acquisition and 
identify gaps in their development. The ePortfolio provides a visual record of progressive 
improvement and achievement of employability capabilities emerging from WIL experiences 
(Simmons & Williams, 2012). 

In addition, there is potential for employers to benefit from student ePortfolios as they 
provide a ready source of evidence to ascertain the suitability of the student's skill set for 
prospective recruitment opportunities. For the vast majority of students, tertiary education is 
a path to gaining meaningful employment in a chosen profession (Halstead & Wheeler, 
2009). The use of ePortfolios supports this aspiration by providing a rich source of 
information for potential employers and professional organizations. WIL pedagogy provides 
an important avenue for transition to the workplace for graduates. The ePortfolio 
complements this experiential approach to student learning by collating electronic evidence 
that demonstrates the complexity and diverse dimensions of graduate capabilities. Such an 
approach augments and consolidates students' proficiencies through providing a holistic 
portrayal of their competencies. 

According to Race and Pickford (2007) assessment is considered the heart of higher 
education, driving student learning and engagement. Furthermore, Hersh (2007) claims that 
designing, executing and marking assessments comprises the majority of the teaching 
academic's workload and is a fundamental component of a quality student experience and 
institutional accountability. According to Edwards and Burnham (2009) ePortfolios provide 
a 'far richer, far more compelling picture of who we are and what we do' (p 89), and may, in 
part, address some of the concerns associated with assessment. The ePortfolio provides 
authentic assessment useful in both formative and summative assessment contexts and is an 
ideal tool for self-assessment (Knight & Yorke, 2004) with the potential to empower students 
to drive their personal learning journey. Given the highly contextual, individualized nature, 
and unpredictable outcomes of WIL assessments, ePortfolios potentially provide a viable 
solution for the problematic nature of assessing WIL. 

DEFINITION 

While there is no one accepted definition for an ePortfolio (also variously known as 
electronic portfolios, webfolios, or e-portfolios) there are some generally agreed upon 
characteristics. Beetham (2006) listed six such characteristics including: a collection of digital 
resources; providing evidence of an individual's progress and achievements, from formal 
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and informal learning activities; personally managed; used for review, reflection and 
personal development; and can be accessed by other interested parties. Sutherland and 
Powell (2007) state ePortfolios are a "purposeful aggregation of digital items . . . with 
evidence of a person's learning and/or ability" (para. 2). Typically an ePortfolio utilizes a 
software product that assists in the organization and display of these digital artifacts that 
may include examples of university assessment tasks, evidence of non-university 
achievements, reflective pieces, and showcase items using a variety of formats including 
graphics, text, video, and sound bites. 

EPORTFOLIOS IN THE HIGHER EDUCATION CONTEXT 

The ePortfolio has emerged within the Australian and global higher education contexts. The 
higher education sector is characterized by a number of key elements which includes the 
growing focus on WIL as an initiative for facilitating work-ready graduates who contribute 
to a productive and robust economy. It is well recognized that student learning and 
competency attainment is not restricted to the university classroom or solely evidenced by 
traditionally assessable tasks of a degree program (Johnsons, Beckerm, Estrada, & Freeman, 
2014). The NMC Horizon Report (Johnsons et al., 2014) attests that society is becoming 
increasingly mobile requiring students to be responsive, agile and creative in their pursuits. 
It is paramount that a university education promotes life-long and life-broad learning in 
response to societal demands (Chen, 2009). Such characteristics are rarely captured in 
traditional academic transcripts, instead requiring artefacts that evidence critical thinking 
and problem solving. 

Increasingly, graduates are globally mobile, requiring competency demonstration that is 
similarly mobile and speaks to an international audience. A recent report indicates that 12% 
of undergraduate students in Australia had an international mobility experience during their 
study program (International Education Advisory Council, 2012). ePortfolios are ideal 
platforms for managing and administering assessment; providing robust and timely 
feedback to students; and maintaining regular and constructive communications when 
students are dispersed throughout the world during the course of their studies. 

An increasing number of professions recognize university entrance pathways and have 
established clear competency frameworks for both initial and ongoing professional 
registration. Some of these professional bodies have opted for an ePortfolio to provide 
evidence for the demonstration of such competencies. An example of this is the accreditation 
requirements mandated by the Australian Nursing Board (Andre, 2010) which includes the 
compilation of student artefacts using an ePortfolio platform. 

Employers have increasingly expressed a desire for potential employees to demonstrate 
generic skills commonly referred to as 'soft skills' such as communication, critical thinking 
and team work. These attributes are perceived as vital skills and complementary to 
professional knowledge and expertise (Jackson & Chapman, 2012; Mitchell, Skinner, & 
White, 2010). Such skills are rarely overtly captured in academic transcripts nor are their 
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acquisition restricted to the classroom environment or traditional assessment tasks (Knight & 
Yorke, 2003). The ePortfolio is an ideal tool for the provision of multiple points of evidence 
that exhibit generic attributes. 

Skill enhancement and achievement is progressed through extra-curricular activities in 
which students participate. While not a formal component of the degree requirements, the 
learning can be equally rich and constructively build on curriculum content. The ePortfolio 
is a ready source of evidence of accomplishments related to both curricula and extra¬ 
curricular activities. Furthermore, it has the potential to present the evidence in a connected 
and integrated way, thereby providing an overarching view of a student's skill set. This is a 
particularly pertinent capacity in a competitive employment market where employers are 
increasingly seeking evidence of community engagement during recruitment. 

AUTHENTIC ASSESSMENT 

Definitions of authentic assessment generally have a common understanding that at its core 
is the integration of theory and practice where assessment tasks are designed to reflect real- 
world scenarios (Shavelson, Klein, & Benjamin, 2009). Wiggins (1993) expressed this as 
"Engaging and worthy problems or questions of importance, in which students must use 
knowledge to fashion performances effectively and creatively". The tasks are either replicas 
of, or analogous to, the kinds of problems faced by adult citizens and consumers or 
professionals in the field” (Wiggins, 1993, p. 229). Mueller (2012) describes this more simply 
as a form of assessment in which students are asked to perform real-world tasks that 
demonstrate meaningful application of essential knowledge and skills. 

It is almost two decades since the term authentic assessment first emerged and debate on 
both refining the definition and its practical implementation continue. Wiggins has more 
recently proposed that 'authenticity' refers to the realism of the setting such that assessment 
should simulate professional doing, not just demonstrating knowledge (Wiggins, 2011). This 
builds on earlier work by Stiggins (1987) in discussing "performance assessments (which) call 
upon the examinee to demonstrate specific skills and competencies, that is, to apply the skills 
and knowledge they have mastered." (Stiggins, 1987, p. 34). 

'Wicked' problems first defined by Horst and Webber (1973) and applied to social planning 
referred to complex, interdependent issues with no simple solutions. Wicked problems 
informed the concept of 'wicked competencies', those competencies that encompass the so 
called 'soft' skills such as creativity, critical thinking and initiative representing complex 
achievements (Knight & Yorke, 2003). Much of WIL seeks to explore and evidence such real- 
world complexity. 

'Wicked' competencies by definition are hard to measure or quantify which tends to result in 
the use of easily administered assessment frameworks (Knight & Yorke, 2003). These authors 
attest that while these competencies are often judged, appraised and evaluated, this does not 
translate into more objective measurement tools. Authentic assessment, performance 
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assessment, and wicked competencies are important concepts to support assessment that 
strives to capture the essence of WIL. 

The ePortfolio has the potential to provide for a less traditional approach to assessment and 
capture assessment outcomes and competencies that are personalized and unpredictable. 
Authentic assessment is pivotal to a broader WIL-based curriculum with the capacity to 
evidence soft or 'wicked' competencies. This is frequently considered as being in opposition 
to traditional approaches to assessment although this should be seen as a continuum rather 
than a strict binary opposition (Mueller, 2012). Drawing on the work of Mueller the key 
elements of these two approaches are summarized in Table 1. 


TABLE 1. Comparing traditional and authentic assessment types 


Traditional 

Authentic 

Selecting a response 

Performing a task 

Contrived 

Real-life 

Recall/recognition 

Construction/application 

Teacher-structured 

Student-structured 

Indirect evidence 

Direct evidence 

Convergent assessment 

Divergent 


Adapted from Mueller (2012). 


Traditional assessment as outlined above is likely to result in a more passive teacher directed 
student learning experience. Authentic assessment requires more active participation by the 
student in a student directed learning experience that is more likely to reflect a real-world 
scenario. 

While ePortfolios provide a platform to capture a student's learning experience in an 
authentic way, recognizing a broad based WIL agenda requires a move from traditional 
assessment. This paradigm shift in assessment is not without tensions. These tensions in 
part arise through such factors as the multiplicity of audience and purpose (Wilhelm et al., 
2006). Table 2 highlights some of these tensions. 

The tensions highlighted in Table 2 may not always be present, however, they present 
challenges for teaching staff who may be discouraged from exploring authentic assessment 
opportunities such as those which use an ePortfolios. The tensions and challenges in the use 
of ePortfolios for assessable tasks are well documented. Shavelson et al (2009) listed the last 
three items in Table 2 as being prominent challenges in the assessment of ePortfolios. They 
encourage an individual, student directed, evidence-based endeavor and as such represent a 
range of experiences and broad learning not restricted to the confines of traditional teaching 
modes. Shavelson et al (2009) argue that this individuality results in a lack of 
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standardization of student productions making cohort comparison, traditionally achieved 
through standardized assessment tasks such as an exam, more challenging. 

TABLE 2. Tensions of using an ePortfolio for student assessment 


Developmental space 

<= 

-► 

For evidence of achievement/showcase space 

Reflective, private space 

<= 

■4 

Public showcase space 

Individually constructed site 

◄- 

-► 

Site architecture defined by institution 

For specific employer viewing 


■4 

Open for any potential employers 

Use to write job applications 

A _ 

_k 

For defined university assessment tasks 


w 

Proscriptive, structured format 

◄- 

-► 

Creative and individual interpretation of use 

Tailored specifically to assist with a 
job application process 

◄- 

-► 

More broadly capturing reflection/developmental 
/assessment work 

For a known and identified and 

invited audience 

◄- 


For an unidentified and unknown audience 

Student engagement driven by 

assessable tasks 

<^= 

=> 

Student self-directed use 

Capturing authentic assessment 

◄- 

-► 

Being used for more traditional assessment 

Accommodation of cultural 

difference and cultural response to 


=^> 

Little scope for cultural difference 

portfolio 




Allows for demonstration of IT 

competency and creativity 


=^> 

Content driven 

Lack of standardization of 

assessment 

«■ 

■4 

Very standardized assessment 

For large-scale class cohorts 


■4 

More individual, student centered approach 

Viewer bias e.g., like the 'look' of 
the portfolio over content 

<= 

■4 

Marking for content only 


Shavelson et al (2009) go on to discuss the challenges of assessing the complexities of diverse 
artifacts that comprise an ePortfolio. The sheer size of some ePortfolios make them time 
consuming and unrealistic to assess for large class groups. Ideally students are given 
freedom to construct and present ePortfolios in an individualized manner with design, photo 
and content decisions made by the student. It is unrealistic to expect assessors to be totally 
unbiased when viewing and assessing these elements. In contrast, traditional marking 
encompasses content as the primary objective to be examined. In concluding, Shavelson et al 
state that these features may render the ePortfolio inappropriate in a tertiary education 
setting. The intention of ePortfolios in an educational setting is that they encompass student 
directed learning which engages students to autonomously reflect, learn, and evidence 
competencies. However, in reality the ePortfolio, like much of tertiary learning, is driven by 
assessable tasks and traditional approaches in higher education settings. Student use of 
ePortfolios is predominantly driven by assessable tasks required for credit rather than 
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student initiated independent work (Woodley & Sims, 2011). Sustaining and maintaining 
independent student engagement is a key challenge for implementing ePortfolios as a 
repository for evidencing authentic assessment outcomes (Shepherd & Skrabut, 2011). 

THE CURTIN EXPERIENCE OF EPORTFOLIOS 

Curtin University's experience in developing and implementing a bespoke university-wide 
ePortfolio system called iPortfolio, are well documented (Oliver, 2010; Oliver, von Konsky, 
Jones, Ferns, & Tucker, 2009; Oliver & Whelan, 2011; von Konsky & Oliver, 2012). In short 
Curtin University's ePortfolio was developed in-house and piloted in 2009. The ePortfolio 
system was introduced to the University community in February 2010. Initial enthusiasm 
resulted in buy-in of some teaching staff who perceived its value in establishing evidence of 
learning achievement for both students and staff. Support was provided to encourage 
widespread adoption which saw it embedded into some teaching and learning practices, 
notably within the common first year of the health science faculty curriculum. The ePortfolio 
was viewed by institutional leaders as a mechanism for responding to a changing tertiary 
educational environment where lifelong and life-broad learning were valued; students 
should be active participants in their own learning and not restricted to classroom-based 
activities; the integration of WIL activities were desirable; and that innovative ways of 
evidencing competencies was essential. The ePortfolio emerged as the Tertiary Education 
Quality and Standards Agency (TEQSA) was established which specified institutional 
requirements for evidencing learning. 

Initially the system was built on several premises, notably that all students would develop an 
ePortfolio and they would recognize the benefits of ePortfolios as a mechanism for 
supporting reflective practice with the ultimate aim of facilitating transition to employment. 
It was anticipated that the inception of an ePortfolio would be embraced by students, staff 
and alumni of the university and expand through a committed core of teaching and learning 
champions as part of teaching innovation underpinned by evidence. Curtin's ePortfolio was 
perceived as a valuable tool for all stakeholders including employers who had the potential 
to access evidence of student proficiency relevant to the profession 

Curtin's Graduate Attributes were supported by a specialist 'my rating' tab in the ePortfolio 
where students rated their level of expertise against each attribute. Through identifying 
when and how they were addressing the Graduate Attributes, familiarity with the generic 
skills improved and students developed expertise in determining what constituted evidence 
of skill acquisition and where they needed to focus their energies for capacity building. 

While there were many successes with the use of the ePortfolio at both an individual teacher 
level (Bathgate, Harris, Comfort, & Oliver, 2011), and at an individual student level 
(Almberg, Comfort, Harris, & Oliver, 2011; Bathgate et al., 2011), from a University-wide 
perspective the success was more limited. The reality of the implementation of the ePortfolio 
system indicates that several of the initial premises were not realized. There were significant 
shifts within the University setting resulting from internal and national drivers that impacted 
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on the success of implementation. The barriers that emerged can be considered under the 
following seven broad categories: approaches to assessment; ePortfolio post-graduation; 
rethinking assessment; staff professional development; technical support infrastructure; 
graduate attributes; and impact of internal and external changes to the educational 
environment. Each of these will now be discussed. 

Approaches to Assessment 

Student and staff use of the ePortfolio was driven by compulsory assessment tasks that 
incorporated mandatory use of the ePortfolio. However, assessment tasks were not always 
well integrated across a subject or a degree program and hence were often conceived as an 
add-on to traditional teaching and assessment approaches and perceived as an isolated 
occurrence rather than a continuum of skill acquisition. A minority of students recognized 
the transferability of the ePortfolio to support transition to employment, however, this was 
not explicitly communicated to students. Consequently, students were unaware of the long 
term benefits of the ePortfolio which ultimately demotivated engagement with the platform. 

The ePortfolio Post-Graduation 

Many students expressed concern about the applicability of the ePortfolio after completion of 
their studies. Despite the system being designed to allow students access to their ePortfolio 
post-graduation, other contemporary portfolio products were considered to have broader 
buy-in and applicability in a global context. Products such as Linkedln 
(http://au.linkedin.com/) were perceived as being more useful, accessible, and conducive to 
establishing and maintaining professional networks as graduate employees. 

Rethinking Assessment 

The lack of a university-wide pedagogy that supported embedding the ePortfolio within the 
broader teaching and learning agenda, impacted negatively on the level of uptake. 
Successful implementation of ePortfolios requires a rethinking of assessment profiles and 
approaches. The ePortfolio platform needs to be embedded across a degree program with a 
developmental approach to learning where experiences are scaffolded across the curriculum. 
Students need to be actively engaged in performance-based tasks where feedback is frequent 
and constructive. While the University promotes an outcomes focused approach to teaching 
and learning, many teaching staff were ill-equipped to make the transition from traditional 
assessment methods to creating and assessing artefacts in an ePortfolio. 

Staff Professional Development 

Integral to successful and sustainable implementation of an ePortfolio system are teaching 
practices that embrace educational philosophies that underpin WIL pedagogy. While 
designing a WIL curriculum is a key feature of authentic learning experiences, the manner 
with which that curriculum is enacted is central to the connectedness and relevance of the 
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learning. During the initial ePortfolio implementation phase, it emerged that many teaching 
staff were unfamiliar with the concept of WIL and how it translates into curriculum and 
integrates with the student experience. Ideally, the roll out of the ePortfolio should be 
coupled with a well-executed professional development program that supports staff in 
integrating the ePortfolio across the curriculum in a holistic and relevant manner. With both 
external and internal agendas promoting advancement of the WIL agenda, it is timely to 
release a contemporary version of the ePortfolio supported by simultaneous WIL 
professional development for teaching staff. 

Technical Support Infrastructure 

The initial ePortfolio project was driven by a partnership between Curtin teaching and 
learning staff and the Information Technology department who were instrumental in the 
final product design. While enthusiasm was evident, the capacity and features of iPortfolio 
were not based on a robust pedagogical understanding. Some identified technical 
inadequacies were not easily remedied resulting in user dissatisfaction. For example, the 
system was unable to deliver sufficient capacity to a large first year cohort; was not 
integrated for use with the learning management system leading to academic staff 
frustrations; and technical limitations meant that uploading of video content was 
problematic. Being an in-house product, maintenance and trouble-shooting was the 
responsibility of the University IT department. However, the capacity to manage the 
considerable workload was not factored into resourcing requirements. Such shortfalls 
hampered the user experience of both students and staff, culminating in reputational 
damage. 

Graduate Attributes 

Curtin Graduate Attributes define student, staff and employer expectations of graduate 
capabilities. As such they cover both discipline specific content and soft or 'wicked' 
competencies and are an imperative component of the ePortfolio system and the WIL 
agenda. They allow for evidence of achievement from both within the classroom and beyond 
- the lifelong, life-broad aspects of the student experience and student learning. In the 
evaluation of the ePortfolio implementation, it emerged that some curricula were ineffective 
in articulating graduate capabilities in outcomes and assessment and failed to recognize the 
nature of adequate evidence to substantiate achievement of outcomes. Assessment tasks 
tended to address individual subject requirements rather than a whole of degree perspective, 
resulting in disconnected and poorly integrated assessment profiles. Typically a strength of 
an ePortfolio and certainly a prerequisite for authentic learning, is the validation of a 
continuum of the development of generic skills. 

Impact of Internal and External Changes to the Educational Environment 

The higher education sector is in the midst of relentless and profound change (Thomas, 
2012). While change presents both opportunities and challenges, the impact on institutions is 
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significant. Institutions respond to external demands by restructuring key organizational 
areas; reshaping teams to address priorities; reallocating staff resources; and refocussing 
strategic directions. These major institutional adaptations geared to meet external 
requirements frequently impact on the continuity and viability of initiatives. As a result of 
reshaping university departments, some dedicated staff were reallocated to other tasks 
which caused gaps in service delivery. It was imperative that support staff and technical 
champions remained stable for continuity of service and valuable corporate knowledge. 

CONCLUSION 

While ePortfolios have been used in the tertiary education sector for many decades, their 
uptake in Australia has not been universal nor has it been without challenges. The case of 
Curtin University illustrates how a desire to evidence students' acquisition of Graduate 
Capabilities and an increasing emphasis on WIL across all teaching areas resulted in the 
design, development and implementation of a custom built ePortfolio system. The outcomes 
from this visionary initiative have been mixed with success and abundant challenges. This 
large scale project highlighted several approaches pertinent to successful implementation of 
an institutional-wide ePortfolio system requiring further investigation, discussion, 
development, resourcing and support. 

There is substantial evidence that ePortfolios provide an ideal platform to evidence program¬ 
wide WIL achievements (Edwards & Burnham, 2009). It provides a setting for students to 
clearly demonstrate that they not only 'know' discipline content but that they can 'do' by 
applying the knowledge in a professional context. This opens up authentic assessment task 
options in preference to traditional assessment methodologies thus providing rich contextual 
demonstrations outside the limitations of customary assessment paradigms. Employers have 
expressed interest in this mode of assessment (Cai, 2012) as they have more confidence in the 
overt demonstration of the achievement of authentic learning experiences which transcend 
the classroom environment. 

While there is clearly a synergy between the use of ePortfolios and WIL, there are potential 
challenges which need to be addressed at an institutional level. First and foremost the 
university community must appreciate and support the relevance and philosophical 
foundations for this shift from knowledge recall to evidencing of achievement that aligns 
with real-world scenarios. Universities need to support WIL practitioners to acquire the 
skills required to incorporate WIL into the student experience and scaffold skill development 
across curriculum. Professional support and adequate resourcing are integral to instilling a 
cultural shift that supports pedagogical approaches for embracing WIL as a student 
engagement strategy. Graduate capabilities aligned to experiential learning outcomes 
evidenced by authentic assessment are indicative of curricula that enhance global 
employability. Traditional assessment profiles are inadequate for assessing experiential 
learning and workplace proficiency. Teaching staff need encouragement to explore new 
approaches where assessment reflects the real-world and has relevance for an emerging 
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professional. Resources in the form of both funding and expertise are critical for promoting 
the use of the ePortfolio. Sustained and innovative leadership is necessary for successful 
development, implementation and evaluation of an ePortfolio platform where there is 
institutional-wide uptake culminating in substantial benefits for students, staff and 
employers. 
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